This section is intended to provide a background or context to the invention that is recited in the claims. The description herein may include concepts that could be pursued, but are not necessarily ones that have been previously conceived or pursued. Therefore, unless otherwise indicated herein, what is described in this section is not prior art to the description and claims in this application and is not admitted to be prior art by inclusion in this section.
Rich media content is generally referred to content that is graphically rich and contains compound (or multiple) media, including graphics, text, video and audio, and is preferably delivered through a single interface. The streaming of rich media content is becoming increasingly important for delivering visually rich content for real-time content, especially within the MBMS and Packet Switched Streaming (PSS) service architecture. Multimedia Broadcast Multicast Service (MBMS) streaming services facilitate resource-efficient delivery of popular real-time content to multiple receivers in a 3G mobile environment. The content may be pre-recorded or generated from a live feed.
There are currently several existing systems for representing rich media, particularly in the Web services domain. Scalable Vector Graphics (SVG) Mobile 1.2 is a language for describing two-dimensional graphics in XML. Scalable Vector Graphics allow for three types of graphic objects: vector graphic shapes (e.g., paths consisting of straight lines and curves), multimedia (such as raster images, video, etc.) and text. SVG drawings can be interactive (using the document object model (DOM) event model) and dynamic. Animations can be defined and triggered either declaratively (i.e., by embedding SVG animation elements in SVG content) or via scripting. Sophisticated applications of SVG are possible by the use of a supplemental scripting language which accesses the SVG Micro Document Object Model (uDOM), which provides complete access to all elements, attributes and properties. A rich set of event handlers can be assigned to any SVG graphical object. Because of its compatibility and leveraging of other Web standards (such as the compound documents format (CDF)), features like scripting can be performed on XHTML and SVG elements simultaneously within the same Web page.
The Synchronized Multimedia Integration Language (SMIL) version 2.0 enables the simple authoring of interactive audiovisual presentations. SMIL is typically used for “rich media”/multimedia presentations which integrate streaming audio and video with images, text or any other media type.
For CDF, there are currently efforts underway to combine separate component languages (e.g. XML-based languages, elements and attributes from separate vocabularies), such as XHTML, SVG, MathML, and SMIL, with a focus on user interface markups. When combining user interface markups, specific problems have to be resolved that are not addressed by the individual markups specifications. These issues include the propagation of events across markups and the combination of rendering or the user interaction model with a combined document.
Real-Time Transport Protocol (RTP) is currently the preferred transport layer protocol for the streaming delivery of continuous media, such as audio, video and SVG. RTP is used for unicast streaming (e.g., 3GPP PSS, 3GPP2 MSS (Multimedia Streaming Services)), broadcast/multicast streaming (e.g., 3GPP multimedia broadcast/multicast service (MBMS), 3GPP2 BCMCS (BroadCast MultiCast Services)) and rich media conferencing applications.
Asynchronous layered coding (ALC) is a massively scalable and reliable content delivery protocol. ALC is the base protocol for the reliable multicast delivery of arbitrary binary objects. ALC is adopted as the mandatory protocol for broadcast/multicast file delivery in 3GPP2 BCMCS and OMA BAC BCAST. The file metadata (that is carried as part of a File Delivery Table (FDT) in file delivery over unidirectional transport (FLUTE)) is now delivered to the clients as part of the OMA BCAST electronic service guide (ESG). This metadata is divided into various ESG fragments according to a service guide data model. The fragments are identified as: Service, Schedule, Content, Access, Session Description, Purchase Item, Purchase Data, Purchase Channel, Service Guide Context, Service Guide Delivery Descriptor and Preview Data. The OMA BCAST ESG data model is illustrated in prior art FIG. 3. The ESG is normally delivered to the clients well in advance of the ALC session. Therefore, clients have the file metadata before the ALC session starts. If the file metadata needs to be updated during the ALC session, then those fragments of ESG that contain the file metadata are updated by using the ESG delivery/update channels. Therefore, file metadata update is not performed in-band of the ALC session.
FLUTE builds on top of ALC and defines FDT that stores the metadata associated with the files being delivered in an ALC session. FLUTE also provides mechanisms for in-band delivery and updates of FDT. FLUTE is adopted by 3GPP MBMS and DVB-H IPDC as the mandatory protocol for broadcast/multicast file delivery.
In addition to the above, There also exists an ISO Base Media File container format for storage of rich media content and subsequent transport of such content over HTTP, FLUTE and RTP. These formats are discussed in detail in U.S. Provisional Patent Application No. 60/713,303, filed Sep. 1, 2005, and U.S. Provisional Patent Application No. 60/694,440, filed Jun. 27, 2005, both of which are incorporated herein by reference in their entirety. However, there are currently no mechanisms for including ALC as a broadcast protocol, particularly for storing file metadata in the ESG.
Until recently, applications for mobile devices were text-based with limited interactivity. However, as more wireless devices are coming equipped with color displays and more advanced graphics rendering libraries, consumers are increasingly demanding a rich media experience from all of their wireless applications. A real-time rich media content streaming service is imperative for mobile terminals, especially in the area of MBMS, PSS, and MMS services.
SVG is designed to describe resolution independent 2D vector graphics, allowing for interactivity using the event model and animation concepts borrowed from SMIL. SVG also allows for infinite zoomability and enhances the power of user interfaces on mobile devices. In addition, SVG supports embedding of media elements similar to SMIL media elements.
All the embedded media can be divided into two parts—discrete media (e.g. images) and continuous media (e.g. audio, video). Continuous media elements define their own timelines within their time container. SVG is therefore, gaining importance and becoming one of the core elements to drive a multimedia presentation, especially for rich media services such as Mobile TV, live updates of traffic information, weather, news, etc. SVG is XML-based, allowing for more transparent integration with other existing web technologies.
For rich media streaming over broadcast/multicast networks, RTP can be used to deliver continuous media such as audio, video and SVG scenes/updates. However and as discussed previously, SVG presentations also include discrete media, and it currently makes more sense to use file download protocols rather than RTP for the delivery of discrete media. ALC and FLUTE are currently the preferred transport layer protocols for file delivery over broadcast/multicast networks. A broadcast/multicast streaming server should be able to generate rich media packets (RTP and ALC, or RTP and FLUTE) by reading the contents from a rich media container file. A container file may include (1) media tracks for continuous media, i.e., SVG tracks, audio tracks, video tracks, raster images, etc.; (2) hint tracks that hold synchronization information; and (3) internally embedded discrete media.
A broadcast/multicast streaming server creates RTP packets to carry continuous media by using the media tracks and hint tracks. The server also needs to create ALC or FLUTE packets to carry internally embedded discrete media. The server may also decide to acquire some or all of the externally referenced discrete media, and send them to the clients using ALC or FLUTE. After reception, these images (a) could be rendered when the corresponding SVG content is being played or (b) can be locally stored/cached and rendered by user interaction. This results in a satisfactory user experience and does not require another simultaneous point-to-point (PtP) connection from the client. However, this optimization does not preclude the possibility of simultaneous PtP connections, but instead merely minimizes the need for simultaneous PtP connections. If these discrete media are not free to access, for example, CNN imagery, then the rich media application can make sure that the images are rendered only if the user buys them (i.e., the digital rights management (DRM) rights). Thus, the externally referenced discrete media files are broadcast to all users whether they subscribed to their access or not. The broadcast download of these files wastes radio and memory resources for users that do not subscribe to these files. For users that subscribe to these files, the broadcast download of these files reduces the usage of radio resources and also enhances their user experience. To create ALC or FLUTE packets, the server needs the metadata associated with the discrete media (image) files. This file metadata also needs to be included in the rich media container file.